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Encoding of two correlated sequences of data. 



FIELD OF THE INVENTION 

The invention relates to encoding a first sequence of data blocks and a 
second sequence of data blocks that is correlated with the first sequence of data 
blocks. The invention may be applied, for exannple, to encode stereoscopic (3D) video. 

5 

BACKGROUND OF THE INVENTION 

It is possible to encode a sequence of data blocks such that certain data 
blocks serve as a reference for predictive encoding the other data blocks. This 
technique is applied, for example, for coding video frames in accordance with a 

10 standard of the Moving Picture Expert Group (MPEG). 

There are applications in which two correlated sequences of data blocks 
need to be coded. The coding of stereoscopic video is an example. Stereoscopic (3D) 
video will generally comprise a' sequence of left-eye video frames and a sequence of 
right-eye video frames. The sequence of left-eye video frames and the sequence of 

15 right-eye video frames can be seen as a first and second sequence of data blocks, 
respectively. The sequences are correlated. 

Stereoscopic video can be encoded in the following manner. The 
sequence of left-eye video frames is encoded in accordance with an MPEG video coding 
technique as if this sequence were an ordinary video signal. This implies that certain 

20 left-eye video frames will serve as a reference for predictive coding the other left-eye 
video frames. As for the sequence of right-eye video frames, each right-eye video 
frame is predictive encoded with respect to the corresponding left-eye video frame. 
That is, each left-eye video frame forms a reference for predictive encoding the 
corresponding right-eye video frame. This manner of encoding stereoscopic video 

25 seems to be disclosed in the European patent application published under number 0 
639 031. 



SUMMARY OF THE INVENTION 

It is an object of the invention to allow a reduction of costs. 
30 The invention takes the following aspects into consideration. In order to 

decode a data block that has been predictive encoded, it is necessary that each data 
block that has served as a reference for this predictive encoding is available. 
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Consequently, at a decoding end, it will be necessary to temporarily store data blocks 
that have served as a reference for predictive encoding other data blocks. 

In accordance with the prior art, each right-eye video frame is predictive 
encoded with respect to the corresponding left-eye video frame. This implies that each 

5 left-eye video frame needs to be temporarily stored at the decoding end. This storage 
will require a relatively large memory in particular because video frames generally 
comprise relatively many bits. 

In accordance with the invention, a first sequence of data blocks is 
encoded such that certain data blocks serve as a reference for predictive encoding the 

10 other data blocks. A second sequence of data blocks, which is correlated with the first 
sequence of data blocks, is encoded such that all data blocks in are predictive coded 
with respect to the data blocks in the first sequence that serve as a reference. 

Accordingly, at the decoding end, it will not be necessary to store all 
data blocks comprised in the first sequence whereas, in contrast, this is necessary in 

15 the prior art. It is sufficient to store those data blocks in the first sequence that served 
3S 3 reference. Consequently, a decoder in accordance with the Invention will require 
less storage capacity than a decoder in accordance with the prior art. Consequently, 
the Invention allows a reduction of cost. 

Another advantage of the Invention relates to the following aspects. 

20 Predictive encoding is inherently entails some inaccuracies: a prediction is rarely 100% 
right. These inaccuracies will generally have an adverse effect on the quality of the 
data that will be obtained at a decoding end. In accordance with the prior art, some 
data blocks in the second sequence are predictive encoded with respect to data blocks 
in the first sequence that are themselves results of predictive encoding. That is, in 

25 accordance with the prior art, the encoding of some data blocks involves two predictive 
encoding in series. Consequently, inaccuracies will accumulate. Since, In accordance 
with the invention, all data blocks In the second sequence are predictive coded with 
respect to the data blocks in the first sequence that serve as a reference, such an 
accumulation of Inaccuracies can not occur. Consequently, the invention allows better 

30 quality. 

These and other aspects of the invention will be described in greater 
detail hereinafter with reference to drawings. 



BRIEF DESCRIPTION OF THE DRAWINGS 
35 FIG.l Is a conceptual diagram illustrating basic features of the Invention 

described hereinbefore; 
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FIG. 2 is a block diagram illustrating an encoder for stereoscopic video in 
accordance with the invention; 

FIG. 3 is a diagram illustrating an encoding of left-eye and right-eye 
video frames in accordance with the invention; 
5 FIG, 4 is a diagram illustrating encoded stereoscopic video data that the 

encoder illustrated in FIG. 3 provides; 

FIG. 5 is a block diagram illustrating a decoder for stereoscopic video in 
accordance with the invention. 



10 EMBODIMENTS OF THE INVENTION 

The following remarks relate to reference signs. Like entities are 
designated by like letter references in all the Figures. Several similar entities may 
appear in a single Figure. In that case, a digit or a suffix is added to the letter 
reference in order to distinguish like entities. The digit or the suffix may be omitted for 

15 convenience or it may be replaced by an asterisk in the case where its value is not 
important (do not care value). This applies to the description as well as the claims. 

FIG.l illustrates basic features of the invention described hereinbefore. 
There is a first sequence [SQl] of data blocks [DBl-*] and a second sequence [SQ2] of 
data blocks [DB2-*]. The second sequence [SQ2] of data blocks [DB2-*] is correlated 

20 with the first sequence [SQl] of data blocks [DBl-*]. The first sequence [SQl] of data 
blocks [DBl-*] is encoded such that certain data blocks serve as a reference [REF] for 
predictive encoding [PE] the other data blocks in the first sequence [SQl]. The second 
sequence [SQ2] of data blocks [DB2-*] is encoded such that data blocks [DB2-*] are 
predictive encoded [PE] with respect to those data blocks [DBl-*] in the first sequence 

25 [SQl] that serve as a reference [REF]. 

The features illustrated in FIG. 1 may be applied, for example, to encode 
stereoscopic (3D) video. In that case, the data blocks illustrated in Fig. 1 may, for 
example, correspond to video frames. The predictive encoding may be based on motion 
estimation and compensation techniques currently applied for MPEG video coding of 

30 non-stereoscopic video. 

FIG. 2 illustrates an encoder for stereoscopic video in accordance with the 
invention. The encoder receives a sequence [VL] of left-eye video frames and a 
sequence [VR] of right-eye video frames and provides, in response, coded stereoscopic 
video data [CSV], The encoder comprises a pre-processor [PREP], a core-encoder 

35 [CENG], a variable-length encoder [VLC], a formatter [FRMT] and a controller [CNTRL]. 
In more detail, the core-encoder [CENG] comprises a motion estimator and 
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compensator [MCE], a subtracter [SUB], a discrete cosine transformer and quantizer - - 
[DCTQ], an inverse discrete cosine transformer and quantizer [IDCTQ], an adder [ADD] 
and a memory [I^EM]. 

The encoder illustrated in FIG. 2 basically operates as follows. The pre- 
5 processor [PREP] modifies the order of the left-eye video frames in accordance with an 
MPEG standard. That is, the order of the left-eye video frames is modified as if these 
video frames belonged to a non-stereoscopic video signal. The pre-processor [PREP] 
combines the left-eye video frames in the re-arranged order, with the right-eye video 
frames. Accordingly the pre-processor applies a multiplex of left-eye and right-eye 
10 video frames to the core-encoder [CENG]. 

The core-encoder [CENG] encodes the left-eye video frames in 
accordance with an MPEG standard as if these video frames belonged to a non- 
stereoscopic video signal. Consequently, the left-eye video frames will either undergo 
an I, P or B encoding depending on their respective positions in the sequence and the 
15 type of MPEG coding that is applied. In contrast, all right-eye video frames undergo a B 
encoding which is a predictive encoding. More specifically, each right-eye video frame 
is predictive encoded with respect to a left-eye video frame that has undergone either 
an I encoding or a P encoding but not with respect to a left-eye video frame that has 
undergone a B encoding. That is, the core-encoder [CENG] encodes each right-eye 
20 video frame as if the video frame were a left-eye video frame destined to undergo a B 
encoding. The core-encoder [CENG] provides quantized coefficients [GF] for each video 
frame, left-eye or right-eye, it encodes. It also provides predictive encoding parameters 
[PP] if the video frame has been predictive encoded. 

The variable-length encoder [VLC] translates the quantized coefficients 
25 [GF] and the predictive encoding parameters [PP] into variable-length code words. The 
formatter [FRMT] combines the code words of different origin and any other data that 
will be required at a decoding end such as identifiers. The formatter [FRMT] casts all 
this data in an appropriate format. Accordingly, the formatter [FRMT] provides the 
encoded stereoscopic video data [CSV]. It goes without saying that the controller 
30 [CNTRL] is suitably programmed so as to provide control signals to the various entities 
in the encoder and cause these entities to function as described hereinbefore. 

FIG.3 illustrates the encoding of left-eye and right-eye video frames. A 
rectangle represents a video frame to be encoded. Four left-eye video frames [L] and 
four right-eye video frames [R] are illustrated. The suffix in the reference sign of a 
35 video frame indicates the position of the video frame in the sequence [VL,VR] that the 
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encdderillustrated in FIG. 2 receives. The letter I, P or B inside a rectangle denotes tfie 
encoding which the video frame concerned will undergo. 

FIG. 3 illustrates predictive encoding by means of arrows. A video frame 
from which an arrow emanates is an video frame that will be predictive encoded. The 
5 video frame at which the arrow terminates is a video frame that serves as a reference 
for this predictive encoding. For example, left-eye video frames L(n+1) and L(n-H2) will 
be predictive encoded with respect to left-eye video frames L(n) and L(n+3). All right- 
eye video frames [R] will also be predictive encoded with respect to left-eye video 
frames L(n) and L(n+3). The pre-processor [PREP] may apply the left-eye and right-eye 

10 video frames to the core-encoder [CENG] in the following order: L(n), L(n+3), R(n), 
L(n + 1), R(n+1), L(n+2), R(nH-2), R(n+3). 

FIG. 4 illustrates the encoded stereoscopic video data [CSV] that the 
encoder illustrated in FIG. 2 provides. The encoded stereoscopic video data [CSV] 
comprises encoded left-eye and right-eye video frames [Lc, Rc] represented as 

15 rectangles. The encoded left-eye and right-eye video frames [Lc, Rc] result from the 
encoding of the left-eye and right-eye video frames [L, R] illustrated in FIG. 3. The 
suffix in the reference sign of an encoded left-eye or right-eye video frame [Lc, Rc] 
indicates the left-eye or right-eye video frame [L, R], respectively, which has been 
encoded. The letter I, P or B inside a rectangle denotes the encoding which has been 

20 applied. For example, encoded left -eye video frame Lc(n+1) is the B-encoded version 
of the left-eye video frame L(n+1) illustrated in FIG. 3. 

FIG. 5 illustrates a decoder for stereoscopic video in accordance with the 
invention. The decoder receives the encoded stereoscopic video data [CSV] illustrated 
in FIG. 4 via a transmission channel that is not shown. In response, it provides a 

25 sequence [VU] of decoded left-eye video frames and a sequence [VR'] of decoded 
right-eye video frames. The decoder comprises a de-formatter [DFRMT], a variable- 
length decoder [VLD], a core-decoder [CDEC], a stereoscopic video demultiplexer 
[STDEMUX] and a controller [CNTRL]. In more detail, the core-decoder [CDEC] 
comprises an inverse discrete cosine transformer and quantizer [IDCTQ,] a motion 

30 compensator [MC] and a memory [MEM], 

The decoder illustrated in FIG. 5 basically operates as follows. The de- 
formatter [DFRMT] separates the data that should be variable-length decoded from the 
data that should not be variable-length decoded. The latter data is applied to the 
controller [CNTRL]. The data may include, for example, identifiers, which the 

35 stereoscopic video demultiplexer [STDEMUX] may use to distinguish between left-eye 
video frames and right-eye video frames. The variable-length decoder [VLD] translates 
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code words back into the quantized coefficients [CF] and the predictive encoding <? * . 
parameters [PP] that have been established in the encoder illustrated in FIG. 2. 

The core-decoder [CDEC] re-constitutes left-eye and right-eye video 
frames on the basis of the quantized coefficients [CF] and the predictive encoding 
5 parameters [PP]. More specifically, the core-decoder [CDEC] reconstitutes the left-eye 
video frames as if they originated from standard non-stereoscopic MPEG-coded video 
data. The core-decoder [CDEC] reconstitutes the right-eye video frames as if they were 
B-encoded left-eye video frames. Referring to FIG. 3, it is sufficient that the memory 
[MEM] temporarily stores the I-encoded video frame L(n) and the P-encoded video 

10 frame L(n+3) in order to decoding all video frames illustrated in FIG. 3. 

The stereoscopic video demultiplexer [STDEMUX] re-arranges the order 
of the left-eye video frames that the core-decoder [CDEC] provides. It further de- 
multiplexes the left-eye video frames and the right-eye video frames. For example, 
referring to FIG. 3, the core-decoder [CDEC] will successively provide decoded versions 

15 of the video frames L(n), L(n+3), R(n), L(n-hl), R(n+1), L(n+2), R(n+2), R(n+3). After 
a certain delay, the stereoscopic video demultiplexer [STDEMUX] will then successively 
provide the decoded versions of the left-eye video frames [L] and, in parallel, the 
decoded versions of the right-eye video frames [R]. Accordingly, the decoded sequence 
[VL'] of left-eye video frames and the decoded sequence [VR'] of right-eye video 

20 frames are obtained. It goes without saying that the controller [CNTRL] is suitably 

programmed so as to provide control signals to the various entities in the decoder and 
cause these entities to function as described hereinbefore. 

The drawings and their description hereinbefore illustrate rather than 
limit the invention. It will be evident that there are numerous alternatives, which fall 

25 Within the scope of the appended claims. In this respect, the following closing remarks 
are made. 

There are various manners to format encoded stereoscopic video. FIG. 4 
illustrates only one possible format in which the order of the encoded frames is Lc(n/I), 
Lc(n+3/P), Rc(n/B), Lc(n+1/B), Rc(n+1/B), Lc(n+2/B), Rc(n+2/B), Rc(n+3/B). Another 

30 possible format is, for example, Lc(n/I), Lc(n+3/P), Lc(n + 1/B), Lc(n+2/B), Rc(n/B), 
Rc(n+1/B), Rc(n+2/B), Rc(n+3/B). Many different formats are possible, just like many 
different formats exist for standard MPEG encoding. 

There are various manners to transfer encoded stereoscopic video from 
an encoder to a decoder. For example, the encoder illustrated in FIG. 2 may be coupled 

35 to the decoder illustrated in FIG. 5 via a transmission channel that carries the encoded 
stereoscopic video data [CSV]. Another possible implementation is, for example, that 
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the enco$ler is part of a recording device, which implies that the encoded stereoscopic 
video data [CSV] is recorded on a carrier. The decoder may be part of the same 
recording device or an other device that retrieves the encoded stereoscopic data [CSV] 
from the carrier. 



of hardware or software, or both. In this respect, the drawings are very diagrammatic, 
each representing only one possible embodiment of the invention. Thus, although a 
drawing shows different functions as different blocks, this by no means excludes that a 
single item of hardware or software carries out several functions- Nor does it exclude 
10 that an assembly of items of hardware or software or both carry out a function. 



claim. The word ''comprising" does not exclude the presence of other elements or steps 
than those listed in a claim. The word "a" or ''an" preceding an element or step does 
not exclude the presence of a plurality of such elements or steps. 



5 



There are numerous ways of implementing functions by means of items 



Any reference sign in a claim should not be construed as limiting the 
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1. A method of encoding a first sequence [SQl] of data blocks [DBl-*] and 

a second sequence [SQ2] of data blocks [DB2-*] that is correlated with the first 
sequence [SQl] of data blocks [DBl-*], the method comprising the step of encoding 
the first sequence [SQl] of data blocks [DBl-*] such that certain data blocks serve as 
5 a reference [REF] for predictive encoding [PE] the other data blocks in the first 

sequence [SQl], characterized in that the method comprises the step of encoding the 
second sequence [SQ2] of data blocks [DB2-*] such that all data blocks [DB2-*] are 
predictive encoded [PE] with respect to those data blocks in the first sequence [SQl] 
that serve as a reference [REF]. 

10 2. An encoder for encoding a first sequence [VL] of data blocks [L] and a 

second sequence [VR] of data blocks [R] that is correlated with the first sequence [VL] 
of data blocks [L], the encoder being arranged to encode the first sequence [VL] of 
data blocks [L] such that certain data blocks serve as a reference for predictive 
encoding the other data blocks [L(n+1), L(n+2)] in the first sequence [VL], 

15 characterized in that the encoder is arranged to encode the second sequence [VR] of 
data blocks [R] such that all data blocks [R] are predictive encoded with respect to 
those data blocks in the first sequence [VL] that serve as a reference [L(n), L(n+3)]. 

3. A decoder for decoding a multiplex [CSV] of a first sequence [VL] of data 

blocks [L] which has been encoded and a second sequence [VR] of data blocks [R] 

20 which has been encoded, the second sequence [VR] of data blocks [R] being correlated 
with the first sequence [VL] of data blocks [L], the decoder being arranged to decode 
the first sequence [VL] of data blocks [L] such that certain data blocks [L(n), L(n+3)] 
serve as a reference for predictive decoding the other data blocks [L(n+1), L(n+2)] in 
the first sequence [VL], characterized in that the decoder is arranged to decode the 

25 second sequence [VR] of data blocks [R] such that all data blocks [R] are predictive 
decoded with respect to those data blocks in the first sequence [VL] that serve as a 
reference [L{n), L(n+3)]. 
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4. A multiplex [CSV] of a first sequence [VL] of data blocks [L] which has" ' 

been encoded and a second sequence [VR] of data blocks [R] which has been encoded, 
the second sequence [VR] of data blocks [R] being correlated with the first sequence 
[VL] of data blocks [L], the first sequence [VL] of data blocks [L] having been encoded 
5 such that certain data blocks [L{n), L(n+3)] serve as a reference for predictive 
decoding the other data blocks [L(n+1), L(n+2)] in the first sequence [VL], 
characterized in that the second sequence [VR] of data blocks [R] has been encoded 
such that all data blocks [R] are predictive encoded with respect to those data blocks In 
the first sequence [VL] that serve as a reference [L(n), L(n+3)]. 
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A first sequence [SQl] of data blocks [DBl-*] and a second sequence 
[SQ2] of data blocks [DB2-*] are encoded. The second sequence [SQ2] of data blocks 
[DB2-*] is correlated with the first sequence [SQl] of data blocks [DBl-*]. This may 
concern, for example, an encoding of stereoscopic (3D) video. The first sequence [SQl] 
5 of data blocks [DBl-*] is encoded such that certain data blocks serve as a reference 
[REF] for predictive encoding [PE] the other data blocks in the first sequence [SQl]. 
The second sequence [SQ2] of data blocks [DB2-*] is encoded such that all data blocks 
[DB2-*] are predictive encoded [PE] with respect to those data blocks [DBl-*] in the 
first sequence [SQl] that serve as a reference [REF]. For example, in a stereoscopic 
10 (3D) video encoding application, left-eye video frames are MPEG-encoded as if they 
belonged to an ordinary, non-stereoscopic video signal and right-eye video frames are 
all B-encoded with respect to I and P-encoded left-eye video frames, or vice versa. A 
relatively small memory will be sufficient at a decoding end for the purpose of 
decoding. 

15 

FIG.l 

I 



10 



31-03-2000 



THIS PAGE BUVMKlwno) 




Printed:ilBM 1-2000 



1-III-PHFR000031 



31-03-2000 



EP00400902.3 



DRAW 



2/3 



|1 I ItJ T 



FRMT 



CNTRL 



■ 






— ^ 






FIG.2 




Printed: 15- 11-2000 



3/3 



B 



5 



B 



B 



FIG.4 




FIG. 5 



3-III-PHFR000031 



THIS PAGE BUNK (uspto) 



